Ensemble model of Artificial Neural Networks with randomized number of hidden neurons

Conventional artificial intelligence techniques and their hybrid models are incapable of handling several hypotheses at a time. The limitation in the performance of certain techniques has made the ensemble learning paradigm a desirable alternative for better predictions. The petroleum industry stands to gain immensely from this learning methodology due to the persistent quest for better prediction accuracies of reservoir properties for improved hydrocarbon exploration, production, and management activities. Artificial Neural Networks (ANN) has been applied in petroleum engineering but widely reported to be lacking in global optima caused mainly by the great challenge involved in the determination of optimal number of hidden neurons. This paper presents a novel ensemble model of ANN that uses a randomized algorithm to generate the number of hidden neurons in the prediction of petroleum reservoir properties. Ten base learners of the ANN model were created with each using a randomly generated number of hidden neurons. Each learner contributed in solving the problem and a single ensemble solution was evolved. The performance of the ensemble model was evaluated using standard evaluation criteria. The results showed that the performance of the proposed ensemble model is better than the average performance of the individual base learners. This study is a successful proof of concept of randomization of the number of hidden neurons and demonstrated the great potential for the application of this learning paradigm in petroleum reservoir characterization

Ensemble model of non-linear feature selection-based Extreme Learning Machine for improved natural gas reservoir characterization

The deluge of multi-dimensional data acquired from advanced data acquisition tools requires sophisticated algorithms to extract useful knowledge from such data. Traditionally, petroleum and natural gas engineers rely on “rules-of-thumb” in the selection of optimal features with much disregard to the hidden patterns in operational data. The traditional multivariate method of feature selection has become grossly inadequate as it is incapable of handling the non-linearity embedded in such natural phenomena. With the application of computational intelligence and its hybrid techniques in the petroleum industry, much improvement has been made. However, they are still incapable of handling more than one hypothesis at a time. Ensemble learning offers robust methodologies to handle the uncertainties in most complex industrial problems. This learning paradigm has not been well embraced in petroleum reservoir characterization despite the persistent quest for increased prediction accuracy. This paper proposes a novel ensemble model of Extreme Learning Machine (ELM) in the prediction of reservoir properties while utilizing the non-linear approximation capability of Functional Networks to select the optimal input features. Different instances of ELM were fed with features selected from different bootstrap samplings of the real-life field datasets. When benchmarked against existing techniques, our proposed ensemble model outperformed the multivariate regression-based feature selection, the conventional bagging and the Random Forest methods with higher correlation coefficient and lower prediction errors. This work confirms the huge potential in the capability of the new ensemble modeling paradigm to improve the prediction of reservoir properties

A least-square-driven functional networks type-2 fuzzy logic hybrid model for efficient petroleum reservoir properties prediction

Various computational intelligence techniques have been used in the prediction of petroleum reservoir properties. However, each of them has its limitations depending on different conditions such as data size and dimensionality. Hybrid computational intelligence has been introduced as a new paradigm to complement the weaknesses of one technique with the strengths of another or others. This paper presents a computational intelligence hybrid model to overcome some of the limitations of the standalone type-2 fuzzy logic system (T2FLS) model by using a least-square-fitting-based model selection algorithm to reduce the dimensionality of the input data while selecting the best variables. This novel feature selection procedure resulted in the improvement of the performance of T2FLS whose complexity is usually increased and performance degraded with increased dimensionality of input data. The iterative least-square-fitting algorithm part of functional networks (FN) and T2FLS techniques were combined in a hybrid manner to predict the porosity and permeability of North American and Middle Eastern oil and gas reservoirs. Training and testing the T2FLS block of the hybrid model with the best and dimensionally reduced input variables caused the hybrid model to perform better with higher correlation coefficients, lower root mean square errors, and less execution times than the standalone T2FLS model. This work has demonstrated the promising capability of hybrid modelling and has given more insight into the possibility of more robust hybrid models with better functionality and capability indices

Improving the prediction of petroleum reservoir characterization with a stacked generalization ensemble model of support vector machines

The ensemble learning paradigm has proved to be relevant to solving most challenging industrial problems. Despite its successful application especially in the Bioinformatics, the petroleum industry has not benefited enough from the promises of this machine learning technology. The petroleum industry, with its persistent quest for high-performance predictive models, is in great need of this new learning methodology. A marginal improvement in the prediction indices of petroleum reservoir properties could have huge positive impact on the success of exploration, drilling and the overall reservoir management portfolio. Support vector machines (SVM) is one of the promising machine learning tools that have performed excellently well in most prediction problems. However, its performance is a function of the prudent choice of its tuning parameters most especially the regularization parameter, C. Reports have shown that this parameter has significant impact on the performance of SVM. Understandably, no specific value has been recommended for it. This paper proposes a stacked generalization ensemble model of SVM that incorporates different expert opinions on the optimal values of this parameter in the prediction of porosity and permeability of petroleum reservoirs using datasets from diverse geological formations. The performance of the proposed SVM ensemble was compared to that of conventional SVM technique, another SVM implemented with the bagging method, and Random Forest technique. The results showed that the proposed ensemble model, in most cases, outperformed the others with the highest correlation coefficient, and the lowest mean and absolute errors. The study indicated that there is a great potential for ensemble learning in petroleum reservoir characterization to improve the accuracy of reservoir properties predictions for more successful explorations and increased production of petroleum resources. The results also confirmed that ensemble models perform better than the conventional SVM implementation

Prediction Of Petroleum Reservoir Properties Using Nonlinear Feature Selection And Ensembles Of Computational Intelligence Techniques

Computational Intelligence (CI) techniques have been applied in the prediction of various petroleum reservoir properties but with ample room for improvement. The major objective of the reservoir characterization process is to provide accurate estimates of the reservoir properties to populate full-field simulation models. The recent use of advanced and sophisticated data acquisition tools has led to a data explosion accompanied by very high dimensional data and increased uncertainties. There is the need for robust techniques that will utilize the strengths of some to overcome the weaknesses of others to produce the best results. Despite the persistent quest for better prediction accuracies in the prediction of petroleum reservoir properties, the application of advanced CI methodologies of hybrids and ensembles has either been slowly embraced or not adequately applied. In this thesis, new non-linear feature-selection assisted methods and ensemble learning models are proposed. The algorithms were implemented with optimized tuning parameters and validated with real-life porosity and permeability datasets obtained from diverse and heterogeneous petroleum reservoirs after they have passed on testing them with a benchmark dataset from the UCI Machine Learning Repository. Common metrics were used to evaluate the performance of the proposed models. The standard machine learning paradigm of dividing datasets into training and testing subsets was employed. When implemented on real petroleum engineering datasets, the proposed Functional Networks-Support Vector Machine assisted model attained the highest R-Square performance of 0.96 and 0.87 on the porosity and permeability datasets respectively (compared to the benchmark of 0.90 and 0.82 respectively) and a total execution time of less than 5 seconds. The ensemble models of Artificial Neural Networks with sequentially searched number of hidden neurons, Support Vector Regression with diverse number of the regularization parameter and Extreme Learning Machine assisted with feature selection and randomized assignment of the number of hidden attained the highest R-Square of 0.99 on the porosity and permeability datasets respectively (compared to the benchmark of 0.89 and 0.90 respectively). A thorough analysis of the comparative results showed that our proposed methods and algorithms outperformed the benchmarks. It was concluded that the proposed assisted and ensemble models will significantly increase petroleum exploration efficiency. A number of hybrid and ensemble possibilities have been recommended for future study

A Hybrid of Functional Networks and Support Vector Machine Models for the Prediction of Petroleum Reservoir Properties

This paper presents an innovative hybrid of Functional Networks and Support Vector Machines (FN-SVM) as an improvement over an existing Functional Networks and Type-2 Fuzzy Logic (FN-T2FL) hybrid model. The former is more promising as it combines two existing techniques that are very close in performance and well known for their computational stability and fast processing. This proposed FNSVM hybrid model benefits from the excellent performance of the least-square-based model-selection algorithm of Functional Networks and the non-linear high-dimensional feature transformation capability that is based on structural risk minimization and Tikhonov regularization properties of SVM. Training and testing the SVM component of the hybrid model with the best and dimensionally-reduced variables from the input data resulted in better performance with higher correlation coefficients, lower root mean square errors and further less execution time than the standard SVM model. A comparison of FN-SVM with the existing FN-T2FL, using the same data and operating environment, showed that the FNSVM is more accurate and consumes less time

Predicting Petroleum Reservoir Properties from Downhole Sensor Data using an Ensemble Model of Neural Networks

The acquisition of huge sensor data has led to the advent of the smart field phenomenon in the petroleum industry. A lot of data is acquired during drilling and production processes through logging tools equipped with sub-surface/down-hole sensors. Reservoir modeling has advanced from the use of empirical equations through statistical regression tools to the present embrace of Artificial Intelligence (AI) and its hybrid techniques. Due to the high dimensionality and heterogeneity of the sensor data, the capability of conventional AI techniques has become limited as they could not handle more than one hypothesis at a time. Ensemble learning method has the capability to combine several hypotheses to evolve a single ensemble solution to a problem. Despite its popular use, especially in petroleum engineering, Artificial Neural Networks (ANN) has posed a number of challenges. One of such is the difficulty in determining the most suitable learning algorithm for optimal model performance. To save the cost, effort and time involved in the use of trial-and-error and evolutionary methods, this paper presents an ensemble model of ANN that combines the diverse performances of seven "weak" learning algorithms to evolve an ensemble solution in the prediction of porosity and permeability of petroleum reservoirs. When compared to the individual ANN, ANN-bagging and RandomForest, the proposed model performed best. This further confirms the great opportunities for ensemble modeling in petroleum reservoir characterization and other petroleum engineering problems

Hybrid intelligent systems in petroleum reservoir characterization and modeling: the journey so far and the challenges ahead

Computational intelligence (CI) techniques have positively impacted the petroleum reservoir characterization and modeling landscape. However, studies have showed that each CI technique has its strengths and weaknesses. Some of the techniques have the ability to handle datasets of high dimensionality and fast in execution, while others are limited in their ability to handle uncertainties, difficult to learn, and could not deal with datasets of high or low dimensionality. The ‘‘no free lunch’’ theorem also gives credence to this problem as it postulates that no technique or method can be applicable to all problems in all situations. A technique that worked well on a problem may not perform well in another problem domain just as a technique that was written off on one problem may be promising with another. There was the need for robust techniques that will make the best use of the strengths to overcome the weaknesses while producing the best results. The machine learning concepts of hybrid intelligent system (HIS) have been proposed to partly overcome this problem. In this review paper, the impact of HIS on the petroleum reservoir characterization process is enumerated, analyzed, and extensively discussed. It was concluded that HIS has huge potentials in the improvement of petroleum reservoir property predictions resulting in improved exploration, more efficient exploitation, increased production, and more effective management of energy resources. Lastly, a number of yet-to-be-explored hybrid possibilities were recommended